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Abstract. This paper shows how proof nets can be used to formalize the notion 
of "incomplete dependency" used in psycholinguistic theories of the unacceptability 
of center-embedded constructions. Such theories of human language processing can 
usually be restated in terms of geometrical constraints on proof nets. The paper 
ends with a discussion of the relationship between these constraints and incremental 
semantic interpretation. 



1. Introduction 

The distinction between competence and performance popularized by 
Chomsky (1965) is fundamental to modern linguistics. Competence is 
the "knowledge" of a human language possessed by its users (e.g., as 
formalized in a grammar), while performance refers to how this knowl- 
edge is used to produce and comprehend utterances. To date, most 
applications of logic to natural language have concentrated on compe- 
tence grammar. This paper suggests that techniques from logic such as 
proof nets (Girard, 1995) might also be useful for describing the diffi- 
culty of processing center-embedded constructions, which is an aspect 
of human linguistic performance. 

It is well-known that center embedded constructions such as (|lb| ) 
are more difficult to comprehend than corresponding right embedded 
constructions (|Ta|) . (The prefix indicates the example is difficult or 
impossible to comprehend). 

(1) a. The patient was cured by the drug [ that was administered by 
the intern [ that was supervised by the nurse ] ]. 

b. *The drug [ that the intern [ that the nurse supervised ] admin- 
istered ] cured the patient. 



* This research was performed while on a sabbatical visit to the Rank Xerox 
Research Centre in Grenoble. I would like to thank Brown and RXRC for making 
this possible, my colleagues who encouraged me to learn about Lambek Categorial 
Grammar and Linear Logic, and Ted Gibson for helpful information about human 
performance on center-embedded constructions. In addition, I would like to thank 
the JoLLI reviewers for their helpful and insightful comments on the first draft of 
this paper. 
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As Chomsky (1965) notes, the difficulty of comprehending ( |iq ) does 
not seem to be due to any single construction in (|Tb|), but rather to 
the way those structures are configured. He argues that (|T§) should be 
treated as grammatical (i.e., possessing a well- formed syntactic anal- 
ysis), but that its incomprehensibility arises from an inability of the 
human sentence processing mechanism to find that analysis. 

Chomsky and Miller (1963) propose that short-term memory over- 
load causes this incompleteness. The strictly right embedded construc- 



tions of the kind (la) can be generated by a right-linear grammar, and 
hence can be recognized by a finite state machine. In contrast, a simple 
application of the pumping lemma for regular sets shows that center 
embedded constructions of the kind ([Ft]) cannot be recognized by a 
finite state machine. 

However, it seems that all such comprehension difficulties cannot 
completely be explained in terms of the memory requirements of a 
parsing automaton. As Chomsky (1965) and others note, the particular 
constructions and their arrangement affect comprehensibility; thus the 
processing difficulty of dT|) relative to (||) may be due in part to the 
fact that in (|TI]) the relative clauses involve object extraction, while in 
( |i~a| ) the relative clauses involve subject extraction. 

The following example, from Gibson and Thomas (1996), shows that 
depth of center embedding alone cannot account for all such processing 
difficulty. ( ]2a| ) and ( |2~bj ) are both doubly embedded structures consisting 
of a relative clause (RC) and a sentential complement (SC), differing 
only in the order of embedding, and most phrase-structure parsing 
automata require the same amount of working memory to accept them 
both. However (pal) is easier to comprehend than (EB). 



(2) a. The chance [ sc that the nurse [ RC who the doctor supervised ] 
lost the reports ] bothered the intern. 

b. *The intern [ RC who the chance [ sc that the doctor lost the 
reports ] bothered ] supervised the nurse. 

The literature contains a number of proposals accounting for a vari- 
ety of comprehensibility differences (Gibson and Thomas, 1996; Lewis, 
1996; Stabler, 1994). These theories of processing overload differ in 
the predictions that they make, this paper does not attempt to choose 
between them, or to develop an alternative account. (Indeed, there 
seems to be considerable uncertainty among theorists as to exactly 
which sentences cause processing overload). 

However, all these proposals share the common feature that they 
posit that the human sentence processing mechanism is sensitive to 
the number and kind of "incomplete dependencies" encountered as 
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the sentence is processed from left to right. The precise configurations 
of dependencies which purportedly lead to processing overload differ 
from theory to theory. Unfortunately, the central notion of incomplete 
dependency, and its relationship both to grammar and to parsing mech- 
anisms, is usually only informally explained in these proposals. 

This paper proposes that resource logics provide the appropriate 
tools to formalize and investigate such incomplete dependencies. The 
next section introduces resource logics and proof nets, and the following 
section uses these to express an account of the processing complexity 
of the examples just discussed. The paper ends with a discussion of the 
relationship between that account and incremental semantic processing. 



2. Resources and Dependencies 

Most, if not all, contemporary linguistic theories are based on a view 
of language in which linguistic entities produce and consume resources. 
For example, a transitive verb is standardly viewed as something which 
requires two noun phrases, its subject and object, to fill its "thematic 
grid," i.e., fill its argument requirements in order to produce a complete 
clause, which can in turn serve as an argument of a higher verb. 

Linguistic theories differ as to exactly what types of resources are 
involved in particular sentences, what structural configurations are 
required for a resource requirement to be filled, exactly what the rules 
for resource accounting are, and what other mechanisms play a role in 
language. For example, versions of Categorial Grammar such as Mor- 
rill (1994) 's Type-Logical Grammar are based solely on (directionally 
sensitive) resource accounting. On the other hand, modern versions 
of Transformational Grammar specify a set of configurations in which 
resource consumption can occur (e.g., feature cancellation occurs in 
a Spec-Head configuration) and provides structural operations (Merge 
and Move) to move linguistic entities into such configurations. 

The notion of "incomplete dependency" adopted in most processing 
models derives from a fundamentally resource based view of language. 
A dependency is a producer-consumer pair, and it is "incomplete" at 
a point in a left to right analysis if exactly one member of the pair has 
been encountered in the portion of the input string seen so far. For 
example, at the point marked ' A ' in the simple transitive sentence @ 
the verb-object dependency is incomplete, since the consumer resource 
(i.e., the verb) has been encountered but the producer resource (the 
object noun phrase) has not. 

(3) Kim kissed A Sandy. 
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Recent work in logic has established a general framework in which 
various kinds of resource sensitivity can be formalized and studied. 
This section informally sketches some of the key ideas, but the reader 
should see Girard, Lafont, and Taylor (1989) and van Benthem (1995) 
for a more thorough introduction to this material. 

Linguistic entities are classified into types, which encode their combi- 
natory capability. For example, the noun phrase Kim might be assigned 
the type NP, and a verb phrase snores might be assigned the type 
NP — > S, indicating that it consumes a resource of type NP to produce 
an object of type S. These types also indicate the type of semantic 
interpretation associated with these phrases: e.g., the verb phrase is a 
function from NP meanings to S meanings. 

Resource logics differ in terms of the kinds of structural sensitivity 
they enforce. For example, a grammar consisting solely of a resource 
logic (e.g., Lambek Categorial Grammar) needs to be sensitive to the 
positions of linguistic entities in the sentence. In such a grammar the 
location of the argument with respect to the functor needs to be spec- 
ified, so the undirected implication '— >' is specialized into a leftward 
looking version '\' and a rightward looking version '/'. For example, a 
transitive verb such as touches might be of type (NP\S)/NP, i.e., an 
entity which consumes an NP to its right (the object noun phrase) and 
then consumes an NP to its left (its subject) to produce a saturated 
sentence. 

Directional sensitivity is one dimension of structural sensitivity. In 
general, the more linguistic detail a resource logic is called upon to 
account for, the more refined its type system needs to be. Because 
the domain of locality of linguistic relationships may vary (e.g., head- 
complement dependencies are more local than WH dependencies), dif- 
ferent types may need to be associated with different structural sensitiv- 
ities. A general framework of multimodal substructural logics has been 
developed for formalizing and investigating these interactions (Moort- 
gat, 1997). 

A proof in a resource logic specifies a "plugging", i.e., it identi- 
fies which objects fill the requirements of which other objects (Girard, 
1995). These dependencies or pluggings determine the semantic inter- 
pretation of the utterance; e.g., via the Curry-Howard correspondence 
between (intuitionistic) proofs and A-terms (Girard, Lafont, and Tay- 
lor, 1989; van Benthem, 1995). 

The job of a grammar is to specify just which pluggings can occur 
in a particular human language, and sophisticated resource logics have 
been developed just for this purpose, as mentioned above. However, for 
the purposes of this paper it is not necessary to exactly identify the 
constraints on grammatical pluggings: indeed, one of the strengths of 
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a dependency based approach is that it does not depend on the precise 
details of the particular linguistic theory involved. Rather, it suffices 
that we can determine the pluggings which have actually occured in 
the particular sentences being studied. These pluggings can usually 
be deduced from fairly general linguistic assumptions (e.g., about the 
valence of verbs) and the interpretation of the sentences themselves 
(which must reflect their pluggings, via the Curry-Howard correspon- 
dence) . 

To keep things simple, all the examples in this paper use Lambek 
Categorial Grammar. This grammatical framework, while sufficient to 
describe the examples presented here, is incapable of describing many 
other important constructions that appear in natural languages. More 
sophisticated grammatical frameworks, such as the multi-modal sys- 
tems described Morrill (1994) and Moortgat (1997), can account for a 
much wider set of natural language examples. However, as the inter- 
ested reader can confirm, the observations presented here about the 
complexity of particular examples also hold in these more complex sys- 
tems (as they must, since these more sophisticated grammars assume 
the same pluggings as presented here). 

2.1. Proof nets 

Proof nets are graphic depictions of proofs, i.e., of the dependencies 
or plugging relationships between entities. In the application described 
here it is necessary to systematically distinguish the inputs to the com- 
binatory process (the lexical items) from the single output produced 
from these inputs (the completed sentence). One way to do this is to 
use an intuitionistic logic, and to use proof nets in which the edges are 
directed. 

A proof net is a directed graph composed of proof net connectives. 
The edges of the proof nets used here are directed (as the logic is intu- 
itionistic) and each edge is labelled with a type. A proof net must satisfy 
structural conditions which depend on the kind of structural sensitivity 
imposed by the logic. Proof nets for Lambek Categorial Grammars must 
satisfy (among other constraints) a planarity condition (Roorda, 1991; 
Roorda, 1992; Lamarche and Retore, 1996). Moortgat (1997) describes 
in detail the structural conditions that correspond to the more general 
class of multimodal resource logics. However, as explained above the 
work presented here requires us only to identify the proof nets asso- 
ciated with particular examples, and does not depend directly on the 
general structural constraints themselves, so they are not presented 
here. 
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Figure 1. The first two columns list the proof net connectives used here, and 
the third column provides the corresponding linear logic proof net connective. The 
top two rows specify "connector schema" which decompose complex types into 
their parts; outward-going arcs indicate resources "provided" by this connector and 
inward-pointing arcs indicate resources it "consumes". The third row presents the 
axiom schema; here A ranges over atomic types. The fourth row presents a non- 
logical axiom permitting a WH phrase to plug an NP requirement. 



The first two columns of Figure [l| lists the proof net connectives 
used in the examples here, and the third column lists the corresponding 
linear logic proof net connective for readers familiar with linear logic. 

We permit two kinds of axiom links. In addition to the identity axiom 
(the wire permitting an A phrase to plug an A requirement) we have a 
non-logical axiom permitting a WH phrase to plug an NP requirement, 
and will indicate its use with a dashed line. (This distinction is one that 
any realistic grammar will have to make, as WH and NP dependencies 
exhibit different structural constraints, even though we do not make 
use of that fact here). 

The proof net connectives decompose the types associated with lex- 
ical entries into the resources that they produce and consume, so each 
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Figure 2. A transitive clause, with vertical bars marking divisions between the 
words. 



lexical entry can be regarded as a partial proof net with incoming and 
outgoing edges labelled with atomic categories. These unpacked lexical 
entries are connected with other words and phrases by axiom links. 

The axiom links identify the dependencies between the lexical item 
and its surrounding material, so the unsatisfied dependencies at any 
point in processing are precisely the axiom links that must be cut in 
order to disconnect the proof net at the point of the cut.Q 

We end this section with some examples. Figure |2| shows the proof 
net for the simple transitive sentence Kim admires Sandy. The vertical 
bars indicate cuts made between the words in this sentence. Each cut 
partitions the words or input resources into two groups, correspond- 
ing to words received at some point of time in processing the input. 
Incremental processing in this framework corresponds to the assump- 
tion that the input seen so far is integrated as much as is possible, but 
the axiom links crossing a cut must be disconnected at the point of 



As an anonymous reviewer points out, in non-commutative logics such as LCG 
a cyclic permutation property arises naturally from one method of extending these 
logics to include negation, although it is by no means necessary, as explained in 
Lamarche and Retore (1996). Such a cyclic permutation of inputs can reduce the 
number of axiom links that need to be cut in order to disconnect the proof net. 
However, these permutations of the input are not directly relevant to the issues 
discussed here. We are specifically interested in cuts that divide the input into the 
words heard before a given point in time, and those that come after that point in 
time, and in general a cut in an arbitrary cyclic permutation of the inputs will have 
not have this property. 

However, at a more general level this observation makes the important point that 
non-incremental interpretation (i.e. processing the input in a different order to that 
in which it was perceived) may be more economical than incremental interpretation, 
in the sense of involving fewer incomplete dependencies during interpretation. To my 
knowledge this possibility has not been explored in the psycholinguistics literature. 
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(NP\NP)/(S/WH) 
who 



Figure 3. The partial proof net associated with an object relative pronoun who. The 
partial proof net associated with a subject relative has the same structure except 
that the outgoing arc labelled WH and the incoming arc labelled S are swapped 
and type labels adjusted accordingly. 



time corresponding to that cut because the resources they connect to 
have not yet been received. In example || exactly one axiom link crosses 
each cut, so there is exactly one unsatisfied NP dependency at each of 
these locations. 

Figure ^ shows the partial proof net associated with a relative pro- 
noun. The analysis of relative clauses used here is simplified, and makes 
obviously incorrect linguistic predictions. It only permits peripheral 
NPs to be "extracted", and does not respect syntactic "island" con- 
straints. NPs are treated as atomic units in this paper, so relative claus- 
es are analysed as NP modifiers. Thus a relative pronoun is analysed as 
something which provides a WH resource to its right and consumes the 
S that it appears in. It also consumes an NP to its left and produces a 
new NP, i.e., it functions as an NP modifier. 

These simplifications do not affect the analysis of the examples pre- 
sented in this paper, but clearly if other constructions were to be anal- 
ysed (such as medial extraction) a more sophisticated grammar would 
be required. Morrill (1994) presents a detailed fragment based on a 
multi-modal logic which permits clause medial extraction and accounts 
for syntactic island phenomena. His grammar assigns a single type to 
relative pronouns, rather than the two types used here. However, his 
grammar assigns proof structures to the examples presented in this 
paper with the same general topological features as the ones presented 
here, and the interested reader can confirm that the complexity differ- 
ences presented below hold in his system as well. 
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Figure 4- A proof net for a relative clause modifying the object. 



3. Proof Nets, Dependencies and Processing 

With the formal tools of proof nets now available, it is relatively simple 
to express many dependency-based theories of human sentence process- 
ing complexity as geometric constraints on proof nets. 

First, consider the iterated object relative clauses shown in (Q), 
which are usually described as right-branching constructions. While 
increased length does reduce acceptability, there does not seem to be 
any significant processing overload associated with such examples. 

(4) a. The patient was cured by the drug [ that was administered by 
the nurse ] . 

b. The patient was cured by the drug [ that was administered by 
the nurse [ who was supervised by the doctor ] ]. 

c. The patient was cured by the drug [ that was administered by 
the nurse [ who was supervised by the doctor [ who was admired 
by the student ] ]. 

Figure |] depicts the proof net for ([Ia|) . For simplicity in this proof net 
the passivised forms was cured by and was administered by are treated 
as transitive verbs, rather than being analysed into their component 
lexical items. At most 2 axiom links cross any cut in the proof nets for 
these examples (either two NP links or an NP and a WH link, depending 
on where the cut lies), which is consistent with processing complexity 
being independent of the number of such constructions involved. 

Now consider the iterated subject relative clauses shown in 
which are classic examples of center-embedded constructions. In con- 



pnhsp.tex; 6/02/2008; 17:43; no v.; p. 9 



10 



Mark Johnson 



~1 

NP 



NP 
J 



NP\NP 



WH £ 
S/WH 



NP (NP\NP)/(S/WH) 
the drug that 



NP 



1 



NP 



NP\S 

L- 



NP 
J 



NP 
the nurse 



f 

(NP\S)/NP 
administered 



1 



NP\S 

L 



NP 



(NP\S)/NP NP 
cured the patient 



Figure 5. A proof net for a relative clause modifying the subject. 

trast to the object relative clauses in (|j), two or more levels of embed- 
ding renders the example virtually incomprehensible. 

(5) a. The drug [ that the nurse administered ] cured the patient. 

b. # The drug [ that the nurse [ that the doctor supervised ] admin- 
istered ] cured the patient. 

c. *The drug [ that the nurse [ that the doctor [ that the students 
admired ] supervised ] administered ] cured the patient. 

Figure [5 depicts the proof net for (^a|) . The maximal cut in the con- 
structions (||) occurs just after the most embedded subject. At depth of 
embedding n the maximal cut crosses 3n + 1 axiom links. The assump- 
tion that the human language processor can only keep track of a small 
number of such incomplete dependencies accounts for the increasing 
ill-formedness as the number of such constructions increases. 

It is sometimes claimed that subject complement clauses such as @ 
are easier to comprehend than corresponding subject relatives (Stabler, 
1994). Figure |6| depicts the proof net for (|^). The maximal cut for this 
example crosses one less axiom link than the corresponding relative 
clause example (|5a|), which is consistent with this putative difference. 

(6) The chance [ that the doctor lost the reports ] bothered the nurse. 

Thus far processing complexity seems proportional to the number 
of axiom links crossed by a maximal cut. The examples (pi), reprinted 
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Figure 6. A proof net for a complement subject example 

here as ([?]), suggest that the matter is more delicate. Proof nets for 
these examples are schematically depicted in Figures ^ and || (only 
links crossed by the maximal cut are shown). 

(7) a. The chance [ sc that the nurse [ RC who the doctor supervised ] 
lost the reports ] bothered the intern. 

b. #The intern [ RC who the chance [ sc that the doctor lost the 
reports ] bothered ] supervised the nurse. 

Gibson and Thomas (1996) present a hypothesis that accounts for 
this difference in acceptability which is expressed in terms of the inter- 
nal states of an unspecified automaton that constructs a parse tree. 
They hypothesise that the human sentence processor overloads very 
quickly when it predicts a category whose features are subsumed by 
the features of one of the categories it is currently in the process of 
completing. In their approach a relative clause introduces a prediction 
for an S node with a WH feature, while a sentential complement intro- 
duces a prediction for an S node without a WH feature. They propose 
that prediction for an S node without a WH feature is subsumed by 



a prediction for an S node with a WH feature, so only (7b) requires 
the introduction of a prediction that is subsumed by the features of a 
category currently being parsed. 

This paper does not attempt to empirically evaluate this hypothesis, 
but merely points out that in the context of the current examples, this 
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the chance that the nurse who the doctor supervised lost the reports bothered the intern 

Figure 7. The schematic structure, including the maximal cut, of the proof net for 
a relative clause embedded within the matrix subject's complement. 
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the intern who the chance that the doctor lost the reports bothered supervised the nurse 

Figure 8. The schematic structure, including the maximal cut, of the proof net for 
a subject complement clause embedded within a subject relative clause. 

hypothesis can be restated as a constraint on the sequence of axiom 
links crossed by any cut in the proof net. To see this, note that Gib- 
son and Thomas (1996)'s category predictions correspond to leftward- 
pointing axiom links in a proof net. Thus there are two S predictions at 
the locations of maximal cut complexity in the proof nets in Figures ^ 
and ||. 

We could directly encode Gibson and Thomas (1996) 's proposal in 
a proof net system by enriching the S type with WH features (possibly 
inherited via some percolation mechanism, as they suggest): in proof 
net terms their hypothesis would be that the human sentence processor 
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overloads if a leftward pointing axiom link is embedded within another 
leftward pointing axiom link labelled with a subsuming type. 

However, because WH-dependencies appear in the proof nets used 
here it is not necessary to enrich the S type with WH features, as the 
relevant S axiom links are easily identified geometrically in the proof 
net (they are the ones introduced by expansion of a S/WH type). In 
the proof nets depicted in Figures [?] and || they are the S axiom links 
adjacent to a WH axiom link. 

4. Incremental Semantic Interpretation 

The previous section has shown that theories of processing stated in 
terms of incomplete dependencies can often be restated as geometric 
constraints on proof nets. Such constraints would be more plausible 
if they can be shown to arise naturally from independently required 
mechanisms or processes, such as incremental semantic interpretation. 

As is well-known, one of the major attractions of resource logic 
accounts of natural language is that the syntax-semantics interface can 
take a particularly simple form, as the Curry-Howard correspondence 
pairs each syntactic operation with a corresponding semantic counter- 
part. 

The idea explored in this section is basically as before: we divide 
the utterance into two parts, and examine the complexity the struc- 
tural relationships between those parts. In the previous sections we 
examined a syntactic measure of complexity — the number of incomplete 
dependencies — and argued that proof nets provide a suitable structure 
for investigating these dependencies. In this section we investigate the 
complexity of the semantic interpretations of the sequence of prefixes 
encountered as the sentence is heard and processed. 

Strong incremental interpretation requires that every prefix of a sen- 
tence form a single semantic entity. Take linguistic objects to be pairs 
X : a of a type X and a A-term a of type X, and let T, A be a string 
of lexical type/A-term pairs with an interpretation (5, i.e., 

r,A h S : (3 

Then an incremental interpretation of the prefix T is an interpolant 
X : a that satisfies the pair of constraints: 

Th X :a and I:a,AhS:/3. 

For many resource logics, including LCG, it is possible to prove that 
such incremental interpretations always exist, i.e., if T, A has an inter- 
pretation then r has an incremental interpretation. 
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A natural question to ask is if there is any relationship between 
the interpolants that function as incremental interpretations and the 
proof net cuts used in the accounts of center embedding processing 
complexity. 

Unfortunately, the relationship between cut set size and the com- 
plexity of the interpolant is not straight-forward, because the inter- 
polant complexity can vary depending on the details of the logic. 

Define the complexity of a type as one plus the number of binary 
connectives appearing in it (thus (NP\S)/NP has complexity 3). Then 
the complexity of an interpolant is never less than the number of axiom 
links crossed by the corresponding cut, since the number of atomic links 
constructed by expanding the interpolant using proof net connectives 
is given by the interpolant's complexity. 

On the other hand, the minimum complexity of an interpolant may 
be greater than the number of atomic links crossed by a minimal cut. 
For example, consider the sequence of LCG types corresponding to a 
transitive clause in a SOV language. 

NP : a, NP : b, A NP\(NP\S) : r (8) 

The minimal cut at ' A ' in a proof net for (||) crosses two NP axiom 
links. However, the smallest LCG interpolant at ' A ' is 

S/(NP\(NP\S)) :A/./(6)(a), 

which has complexity 4. 

Interestingly, this difference disappears if we enrich LCG's type with 
product '(8>', with pair formation '(•, •)' as the corresponding semantic 
operation. With this extension the minimal interpolant is 

NP <g> NP : (a, b) 

which corresponds directly to the axioms crossed by the minimal cut. 

Thus the presence of an additional logical connective, in this case 
'<8>', can alter the interpolant complexity, even if that connective is not 
used in the grammar itself. Thus interpolant complexity depends on 
features of the logic used. 

5. Conclusion 

This paper has shown that proof nets provide a suitable formalization 
of the notion of "incomplete dependency" used in many accounts of 
processing complexity. The structure of a proof net for a sentence rele- 
vant for the complexity metrics discussed in this paper is usually fixed 
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by its meaning and standard linguistic assumptions about the types of 
the words involved, so the results obtained do not depend crucially on 
any particular linguistic theory More sophisticated models of process- 
ing, such as Gibson and Thomas (1996), appeal to the arrangement 
of embedding relationships in a sentence. The grapical nature of proof 
nets makes it simple to express such theories in this framework. 

The last section of this paper investigated the relationship between 
the incomplete dependencies expressed by proof nets and the complex- 
ity of the types of incremental semantic interpretations. However, it 
seems that the complexity of the interpolants involved in incremen- 
tal interpretation depend crucially on the details of the logic used to 
represent these interpolants. 
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